An Alternative View of Variational Bayes and Minimum Variational Stochastic Complexity

Bayesian learning is widely used in many applied datamodelling problems and is often accompanied with approximation schemes since it requires intractable computation of the posterior distributions. In this study, we focus on the two approximation methods, the variational Bayes and the local variational approximation. We show that the variational Bayes approach for statistical models with latent variables can be viewed as a special case of the local variational approximation, where the log-sum-exp function is used to form the lower bound of the log-likelihood. The minimum variational stochastic complexity, that is the objective function of the variational Bayes, is also examined and related to the asymptotic theory of Bayesian learning.